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ABSTRACT 



Qualitative evaluation can be used to facilitate 



change in the immediate higher education professional environment. 
This evaluation method enriches the collection and interpretation of 
data by involving all groups and persons who have a share in the 
problem. In contrast to qualitat-^^e research, qualitative evaluatiOii 
assists m situation-specific decision-making and attempts to involve 
all individuals and groups associated with the problem in all phases 
of data collection and interpretation. A model for qualitative 
(^valuation used by Guba and Lincoln consists of nine steps: (1) 
identify the problem (any individual or member of a group can 
Identify a problem); (2) identify shareholders (identify all who are 
associated with or could be effected by the issue); (3) focus 
responsively (generating alternative problem statements); (4) 
prioricii:,e questions, concerns, and issues (group representatives 
together discuss the questions generated m Step 3); (5) identify 
information and identify criteria (deciding what data to collect, 
choosing criteria to evaluate the data) ; (6) collect data (data 
collection not be limited to a single instrument, method or 
person/group); (7) present information anc? prepare agenda for 
negotiation (each group prepares an agenda concerning their preferred 
solution); (8) negotiate solutions (group representatives together 
arrive at a compromise solution); and (9) develop report or written 
action plan* (JB) 



* Reproductions supplied by EDRS are the best that can be made 

* from the original document. 



Qualitative Evaluation in Higher Education 



Qualitative evaluation is a process that goes beyond 
^getting the facts' through qualitative collection and 
interpretation of data. The collection and interpretatioi; 
of data is ^enriched' by involving all the human, political, 
social, cultural and contextual elements of a situation 
(Cuba and Lincoln, 1989). These elements are drawn into liio 
process by using qualitative research methods for data 
collection and by involving all groups and persons who hiw < 
a * share' in the problem. These groups or individuals art 
known as shareholders because they have a vested interest xn 
the outcome or will be effected by the outcome a project.* 

Qualitative evaluation differs from qualitative 
research in two ways. 

1. When a person has questions about the environment 
in general, then qualitative research should be utilized. 
However, when a person has a question which is specific to 
his/her own environment, then qualitative evaluation should 

w 

be used. Qualitative research facilitates the 
generalizations of research findings to various environmeiitir. 
while qualitative evaluation assists in situation-specific- 
decision-making. 

*Shareholders is a modification of the term Stakeholders 
which was used by Cuba and Lincoln (1989). 
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2. Qualitative research limits the input and the 
interpretation to select individuals or groups. Qualitative 
evaluation attempts to involve all individuals and groups 
associated with the problem or concern in all phases of data 
collection and interpretation. 

The focus of this paper is to explain how qualitative 
evaluation can be utilized to facilitate change in the 
immediate professional environment. The following model of 
qualitative evaluation is an adaptation of a model uised by 
Guba and Lincoln (1989). Figure 1 illustrates the nine 
steps which are involved in this qualitative evaluation 
model . 

ste p li Identif y t:he Problem 

in this model any individual or group can identify a 
concern or problem that they would like to address throagh 
the use of this method. This method does not advocate the 
"top-down" problem identification; i.e., chief 
administrators do not ^give' a problem to the people working 
for them. For the purpose of this paper, assume that a 
faculty member wants to address the following issue; How can 
I get my students to increase discussion in class? 
step 2; Identify Shareh olders 

Once an issue is raised, it becomes necessary to 
identify all constituent groups who are associated with or 
could be effected by the decisions made concerning this 
issue. These groups become shareholders and work 
independently of and, in conjunction with, the other groups 



in order to resolve the issue. The faculty member must 
identify all persons who are associated with his/her concern 
about class discussion. While there may be many 
shareholders involved with this problem, for the sake of 
simplicity, let us assume that there are three groups of 
shareholders; the students, the faculty and the 
administrators . 
Step 3: Responsive Focusing 

After all shareholders are identified, they are asked 
to refine the initial problem. The shareholders are asked 
to put ;»side the common assumption that they already know 
that the problem is, and in so doing, they are asked to 
generate alternative problem statements. This generation of 
alternative questions is crucial in order not to preclude 
any possibi] ities and not to precondition the mind to accept 
a narrow range of acceptable solutions to that problem. 

Since the initial problem was "How can I get my 
students to increase class discussion?", the shareholders 
attempt to create que'-^tlons related to this initial problcin. 
For example, the student shareholders may come up with a 
list of questions similar to: 

1. How can class be made more varied? 

2 . How can class be made more relevant? 

3. How can we have more input into course content? 



I 



The faculty shareholders may come up with a list of 
questions similar to: 

1. How can students become active agents in the 

©(^ucational process? 

2. How can we create an environment in which students 

feel confident and comfortable enough to talk? 

3. How can we structure a.ssignments to increase 

students' interest? 
The administrator shareholders may come up with a list 
of questions similar to: 

1. How can we offer a variety of experiences with 

limited personnel and funding? 

2. How can we bring in more guest speakers in classes? 

3. How can we reschedule faculty? 

Step 4; Prioritize Questions. Concerns, and Issues 

It is important to remember that all input from 
shareholders must receive equal weighting and consideration; 
i.e., the input from students must be given the same 
priority as the input from faculty. The groups prioritize 
their list of questions and then representatives from each 
group join together and discuss all of the questions which 
were generated from their individual groups. The 
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combined group would *2xamine the questions in an attempt to 
determine: 

1. VThat's the question about which ideas are most 

needed? 

2. If some questions address similar themes or 

concerns? 

3. If there is a * common strand' among the questions? 

4. On what questions should we start working? 

5. What questions suggest the most useful directions? 
In our example, the faculty/student/ administrator group 

may decide that the student's #3 question (How can we have 
more input in the course content?) and the faculty's #1 
question (How can students become more active agents in the 
educational process?) are essentially addressing the same 
issue. Similarly, the faculty's #3 question (How can we 
structure assignments to increase student interest?) and the 
student's #2 question (How can class be made more relevant?) 
are similar. The group should look for common, overlapping 
concerns. Additionally, the group should select issue (s) 
which serve all the shareholders and not just one * select' 
group. With some discussion and debating the group arrives 
at one question to consider. The group in this example 
decides to combine a question from the student's list with a 
question from the faculty *s and the administrators' list. 
The problem they chose to consider is "How can the, learning 
environment be structured to be more interesting and 
relevant?" 




step 5; Identify Information and Identify Criteria 

After the issue has been refined and selected, two 
other decisions need to be made by the entire group of 
shareholders. The first decision relates to the type of 
data to be collected. Based on the chosen problem, 
information to be collected which would be useful in 
addressing this issue may include such things as: 

1. What class topics do students think are irrelevant? 

2. How much money is available to bring in guest 

speakers? 

3. What topics do other university professors include 

in this type of course? 

4. What teaching methods do the students believe 

assist in their learning of class material? 

5. What techniques are available to make certain 

topics more interesting to students? 
The second decision which the entire group needs to 
make focuses on the selection of criteria by which the data 
can be evaluated. The criteria helps the group determine 
which ideas are truly promising and which ones are weak. 

To return to the example, once the group has decided 
on the ways in which data will be collected, they need to 
select criteria to evaluate these data. The criteria which 
the group might select, in this example, include: 

1. Time to implement idea or time to complete task 

2 . Cost 

3. Availability of personnel, facilities and equipment 
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step 6t Collect Data 

The groups are asked to assist in the data collection 
through participate; observation and through other research 
methods used in crnjunction with participate observation. 
Whatever methods are used proper research procedures need to 
be followed when collecting this data. For example if 
interviews are to be done, then a structured interview 
process must be constructed. If a representative group is 
to be used, instead of all group members, then proper 
sampling procedures need to be followed. It should be 
remember that the means to collect data should not be 
limited to a single instrument, method or person/group. In 
this example, the data which is to be collected includes: 

1. Interviews with students and faculty from various 

classes. 

2. Surveys cf physical t iucation majors from other 

college campuses. 

3. Brainstorming sessions by shareholder groups. 

4. Observations of faculty from other 

university. 



Step 7; Present Information and Prepare Agenda for 
Negotiation 

After all the data has been gathered, the independent 
groups of shareholders review data and then make some 
decisions about the data using the criteria as a guide. 
Each group of shareholders prepares an "agenda" or develops 
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a plan which includes the group's preferred solution to the 
issue. It is essential that the data be superimposed on the 
criteria and that all groups review the data. Students 
will, in all probability, interpret the information much 
differently than faculty or administrators. This difference 
in interpretation will lead the groups to find very 
different and workable solutions to the problem. 
Step 8: Negotiate Solutions^ Concerns and Issues 

The next step in this qualitative evaluation process 
involves bringing shareholding group representatives back 
together in order to develop a mutually agreeable plan which 
would address the issue. The groups are not trying to 
convince each other of the brightness' of one group's 
solution over the other but are attempting to work out a 
compromise. The compromise should incorporate the best 
ideas from all shareholders. 

Step 9: Develop Report or Written Action Plan 

The report is distributed to all shareholders and 
contains chree things: 

1. The action or actions to be taken to resolve the 

issue. 

2. The time frame to accomplish this action. 

3. The person or persons who are responsible for 

completing the tasks. 
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Conclusion 

A man who, after a night of overindulging, attempted to 
unlock his front door. But as luck would have it, just as 
he had the key to the lock, the keys slipped from his hand, 
hit the steps and fell beneath the stairs. The man stumbled 
down the stairs and went out to the street and began 
searching for his keys. A passerby stopped and asked the 
man what he was doing. "I dropped my key and I can't find 
them." The passerby offered his assistance and asked the 
man approximately where he dropped the keys. "Under the 
stairs by my front porch." Amazed that the man was seeking 
his keys out by the street, he asked, "Why are looking for 
your keys out here?" The man looked at the passerby and 
said, '"because the lights better out here?" 

How unlike this man with the lost keys are we in our 
search for solutions to our problems? We typically search 
in the same ways and use the same methods because the "light 
is better there." The use of the qualitative evalaation 
process is an alternative method to seek and to find 
workable solutions which address the concerns of all people 
who are influenced by the problem. 
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